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PREFACE 


This  Note  contains  a  research  design  developed  by  RAND  to  assess  the 
performance  of  Army  personnel  in  operating  the  PATRIOT  air  defense  missile  system. 
The  overall  purpose  is  to  improve  the  ability  of  the  Army  Training  and  Doctrine 
Command  (TRADOC)  to  set  appropriate  performance  standards  and  training  resource 
needs  and  to  respond  to  Congressional  requests  for  quantitative  evidence  linking 
personnel  quality  with  Army  operational  performance. 

The  study  is  one  of  several  TRADOC-sponsored  research  efforts  on  soldier 
performance.  The  RAND  Arroyo  Center  is  conducting  related  studies  in  the  areas  of 
communications  and  electronic  system  maintenance  under  its  Manpower,  Performance, 
and  Training  Program.  In  addition,  TRADOC,  the  U.S.  Military  Academy,  and  the  Army 
Research  Institute  arc  organizing  similar  studies  examining  armored  vehicle  and  artillery 
operations. 

This  Note  is  intended  to  document  the  design  of  an  in-progress  study.  As  data 
become  available,  it  will  be  supplemented  with  RAND  analytic  reports  including 
statistical  estimation  of  relationships  and  models  of  the  implications  of  those 
relationships  for  battlefield  operations. 

The  Arroyo  Center 

The  Arroyo  Center  is  the  U.S.  Army’s  Federally  Funded  Research  and 
Development  Center  for  studies  and  analysis  operated  by  The  RAND  Corporation.  The 
Arroyo  Center  provides  the  Army  with  objective,  independent  analytic  research  on  major 
policy  and  management  concerns,  emphasizing  mid-  and  long-term  problems.  Its 
research  is  carried  out  in  five  programs:  Policy  and  Strategy  Studies;  Force  Development 
and  Employment;  Readiness  and  Sustainability;  Manpower,  Training,  and  Performance; 
and  Applied  Technology. 

Army  Regulation  5-21  contains  basic  policy  for  the  conduct  of  the  Arroyo  Center. 
The  Army  provides  continuing  guidance  and  oversight  through  the  Arroyo  Center  Policy 
Committee,  which  is  co -chaired  by  the  Vice  Chief  of  Staff  and  by  the  Assistant  Secretary 
for  Research,  Development,  and  Acquisition.  Arroyo  Center  work  is  performed  under 
contract  MDA903-86-C-0059. 
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The  Arroyo  Center  is  housed  in  RAND’s  Army  Research  Division.  The  RAND 
Corporation  is  a  private,  nonprofit  institution  that  conducts  analytic  research  on  a  wide 
range  of  public  policy  matters  affecting  the  nation’s  security  and  welfare. 

Stephen  M.  Drezner  is  Vice  President  for  the  Army  Research  Division  and 
Director  of  the  Arroyo  Center.  Those  interested  in  further  information  concerning  the 
Arroyo  Center  should  contact  his  office  directly: 

Stephen  M.  Drezner 
The  RAND  Corporation 
1700  Main  Street 
P.O.  Box  2138 

Santa  Monica,  CA  90406-2138 
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SUMMARY 


The  PATRIOT  missile  system,  the  most  modem  and  automated  of  the  Army’s  Air 
Defense  Artillery  systems,  protects  U.S.  and  NATO  assets  from  the  high-  to  medium- 
altitude  enemy  air  threat.  This  Note  describes  a  research  design  to  assess  the 
performance  of  PATRIOT  enlisted  operators  and  to  link  that  performance  to  the 
outcomes  of  simulated  air  battles.  The  study  is  one  of  several  research  efforts  sponsored 
by  TRADOC  to  develop  quantitative  analyses  based  on  objective  measurement  of  soldier 
and  unit  performance.  The  overall  purpose  is  to  improve  the  Army’s  ability  to  set 
appropriate  performance  standards  and  training  resource  needs  and  to  respond  to 
Congressional  requests  for  quantitative  evidence  linking  personnel  quality  with  Army 
operational  performance. 

The  basic  building  block  of  the  PATRIOT  system  is  a  firing  battery  or  "fire  unit," 
which  includes  missile  launchers,  radar  and  communications  equipment,  and  an 
Engagement  Control  Station  (a  mobile  shelter  containing  display  screen*, 
communications  facilities,  and  computer  consoles).  During  an  air  battle,  the  key 
engagement  decisions  in  a  PATRIOT  battery  are  made  by  one  officer  (Military 
Occupational  Specialty  14E)  and  one  enlisted  person  (MOS  24T),  who  operate  the 
Engagement  Control  Station.  At  battalion  level  (the  next  higher  echelon),  a  comparable 
officer-enlisted  pair  operate  a  similar  facility,  which  controls  the  activities  of  three 
batteries. 

At  battery  level,  MOS  24T  personnel  help  the  officer  by  playing  the  role  of 
Tactical  Control  Assistant  (TCA);  at  battalion  level,  they  play  the  role  of  Tactical 
Director  Assistant  (TDA).  The  research  will  examine  the  efficiency  and  effectiveness 
with  which  the  TCA  and  TDA  are  able  to: 

•  Protect  valuable  assets,  engage  enemy  aircraft,  and  assist  in  the  protection  of 
friendly  aircraft  as  is  required  for  success  during  air  battles  (TCA);  and 

•  Direct  fire  units  to  engage  specific  aircraft  in  order  to  protect  assets  and 
destroy  enemy  aircraft  (TDA). 
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The  study  examines  how  differences  in  personnel  quality  and  training  background 
affect  the  execution  of  TCA  and  TDA  functions  and  the  outcomes  of  air  battles.  All  of 
the  Army's  MOS  24T  soldiers  at  skill  levels  1  and  2  will  be  tested,  including  students 
finishing  Advanced  Individual  Training  (AIT)  and  the  members  of  all  PATRIOT  units 
stationed  both  in  the  United  States  and  in  Europe.  We  estimate  that  up  to  300  unit 
personnel  and  150  students  will  be  tested  during  the  study  period,  from  December  1988 
through  June  1989. 

The  study  will  use  the  PATRIOT  Conduct  of  Fire  Trainer  (P-COFT),  a  computer- 
controlled,  high-fidelity  simulation  facility  consisting  of  eight  operator  consoles.  The 
consoles  can  be  operated  independently  to  simulate  autonomous  fire  unit  operation  or  in 
a  netted  configuration  to  simulate  fully  interactive  battalion  operation. 

Four  20-minute  air  battles  will  be  simulated.  The  four  scenarios  have  been 
developed  especially  for  this  test:  an  area  defense  scenario,  a  point  defense  scenario,  a 
mixed  (area  and  point  defense)  scenario,  and  a  battalion  scenario.  The  scenarios  include 
realistic  wartime-related  operations  and  events  and  provide  a  meaningful  test  of  the 
performance  of  air  defense  missions  for  both  AIT  students  and  unit  members.  In 
addition,  unit  personnel  will  complete  a  written  test  measuring  their  knowledge  of  proper 
tactical  operating  procedures,  and  all  examinees  will  complete  a  test  on  system 
initialization  procedures. 

Two  personnel  will  be  tested  simultaneously.  After  a  briefing  describing  the 
purpose  and  procedures  of  the  test,  they  will  be  asked  to  complete  a  short  background 
questionnaire.  They  will  then  be  tested  on  the  simulated  air  battles  and  written 
examinations.  Just  before  the  beginning  of  each  scenario,  the  examinee  will  be  provided 
with  written  instructions  concerning  his  mission,  the  applicable  air  defense  conditions, 
and  specified  tactics.  The  order  of  written  test  completion  and  air  battle  simulation  will 
be  reversed  for  the  two  examinees,  so  that  they  never  participate  in  the  same  component 
of  the  test  at  the  same  time. 

The  P-COFT  facility  computer  will  automatically  tabulate  all  air  battle  switch 
action  and  outcome  records.  We  plan  to  concentrate  on  four  overall  measures  of 
effectiveness,  representing  success  in  the  missions  of  TCA  point  defense,  TCA  area 
defense,  TDA  operations,  and  identification  of  friends  and  foes  (IFF).  Additional 
supporting  measures  will  include  compliance  with  tactics  and  doctrine,  compliance  with 
communication  received,  and  knowledge  of  initialization  procedures. 
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The  study  plan  outlines  specific  test  factors  (e.g.,  damage  to  each  asset,  hostile 
aircraft  destroyed,  total  missiles  used,  execution  of  cease  fire/engagement  commands) 
that  will  be  available  to  provide  the  appropriate  measures  of  effectiveness  and 
performance.  We  intend  to  model  the  effects  of  various  soldier  characteristics  on  these 
performance  measures,  assessing  the  effects  of  such  variables  as  aptitude  scores, 
educational  background,  training  history,  and  previous  assignments  in  PATRIOT  and 
other  air  defense  jobs. 
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I.  INTRODUCTION 


This  Note  describes  plans  for  research  to  assess  the  proficiency  of  enlisted 
personnel  who  help  to  operate  the  PATRIOT  air  defense  missile  system.  The  tasks 
performed  by  these  soldiers,  who  are  classified  in  Military  Occupational  Specialty 
(MOS)  24T,  are  necessary  to  engage  enemy  aircraft  and  to  protect  friendly  assets  during 
combat. 

The  PATRIOT  is  the  Army’s  most  advanced  air  defense  system,  designed  to  meet 
the  medium-  to  high-altitude  air  threat  of  the  1980s  and  beyond.  It  is  capable  of 
simultaneously  engaging  multiple  targets  in  a  highly  saturated  air  environment,  using 
advanced  features  in  its  radar,  missile  guidance,  and  computer  and  automation  systems.1 
The  basic  building  block  of  the  PATRIOT  system  is  a  firing  battery  or  "fire  unit,"  which 
includes  missile  launchers,  radar  and  communications  equipment,  and  an  Engagement 
Control  Station  (a  truck-mounted  shelter  containing  display  screens,  communications 
facilities,  and  computer  consoles).  During  an  air  battle,  the  key  engagement  decisions  in 
a  PATRIOT  battery  are  made  by  one  officer  (Tactical  Control  Officer — TCO)  and  one 
enlisted  person  (Tactical  Control  Assistant — TCA)  who,  performing  discrete  tasks, 
operate  the  Engagement  Control  Station.2  A  similar  officer-enlisted  pair  play  the  main 
roles  at  the  next  higher  echelon,  in  a  PATRIOT  battalion  composed  of  three  batteries.3 
This  study  aims  to  evaluate  the  abilities  of  these  enlisted  personnel  and  the  effects  of 
their  performance  on  the  outcomes  of  simulated  air  battles. 

There  are  several  reasons  for  assessing  the  performance  of  PATRIOT  system 
operators: 


•  To  derive  statistical  estimates  of  the  effects  of  personnel  quality  and 
experience  on  the  outcomes  of  air  battles, 


!Key  features  are  the  multifunction  phased  array  radar,  track-via-missile  guidance, 
and  computer-automated  operations  to  process  information  and  control  missile  launches. 

2The  station’s  crew  also  includes  a  communications  operator  in  MOS  31M,  but  that 
person  is  not  directly  involved  in  air  battle  operations. 

3Eventually,  six  batteries  will  be  fielded  per  battalion. 
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•  To  ensure  the  results  apply  to  a  modem  air  defense  system  operating  in  a 
wartime-like  environment, 

•  To  provide  test  measures  directly  linked  to  air  battle  outcomes  (asset  damage 
and  aircraft  engagements),  and 

•  To  assess  the  effects  of  personnel  quality  for  both  newly  trained  personnel 
and  experienced  unit  members. 


BACKGROUND  AND  OBJECTIVE 

In  the  past  few  years,  the  U.S.  Army  has  acquired  modernized  equipment  with  the 
potential  to  greatly  expand  unit  combat  capability.  Realizing  that  potential,  however, 
depends  on  the  quality  of  the  Army’s  people  and  the  training  opportunities  they  receive. 
In  this  decade,  the  Army  has  enjoyed  unprecedented  levels  of  quality  among  recruits  and 
increased  levels  of  seniority,  experience,  and  training  among  its  more  senior  personnel.4 
But  emerging  constraints  on  Defense  budgets  are  likely  to  limit  the  Army’s  future  ability 
to  secure  the  numbers  of  high-quality  recruits  to  which  it  has  become  accustomed  and  to 
maintain  current  levels  of  training  resources. 

Indeed,  decisionmakers  in  the  Army,  OSD,  and  the  Congress  are  increasingly 
scrutinizing  recruiting  and  training  budgets.5  To  establish  resource  needs  credibly,  and  to 
most  effectively  utilize  the  resources  that  are  allocated,  the  Army  needs  reliable, 
quantitative  analyses  of  soldier  performance  and  its  determinants,  including  such  broad 
categories  of  training  resources  as  personnel  quality,  opportunity  to  train  on  tactical 
equipment,  and  access  to  training  devices  and  simulators. 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  has  primary 
responsibility  for  setting  the  standards  for  unit  and  individual  training  and  for 
determining  the  required  levels  of  aptitude  and  performance  among  trainees.6  To  set 

4Departmcnt  of  Defense,  1985. 

5For  example,  the  House  Appropriations  Committee,  in  its  report  accompanying  the 
FY  88  military  appropriations  bill,  directed  the  Department  of  Defense  to  develop  new 
methods  of  linking  the  educational  background  and  aptitudes  of  recruits  to  the  ability  of 
units  to  perform  their  operational  missions. 

draining  standards  are  established  in  various  TRADOC-published  Soldier’s  Manuals, 
ARTEPS  (Army  Training  and  Evaluation  Programs),  and  related  publications. 

TRADOC  schools  set  minimum  aptitude  entry  standards  for  their  MOS  courses,  and 
TRADOC  Pamphlet  601-1  establishes  "distribution  of  quality"  criteria  for  allocating 
high-quality  recruits  across  functional  areas. 
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these  standards,  TRADOC  needs  improved  measures  of  soldier  performance  based  on 
objective,  quantified  assessments  of  individual  and  unit  mission  achievement  in 
wartime-related  functions. 

In  the  past,  most  analyses  of  personnel  quality  requirements  have  been  carried  out 
by  TRADOC  schools  and  have  been  based  on  the  minimum  aptitude  levels  that  recruits 
need  to  pass  initial  skill  training  courses.7  The  Army’s  long-range  job  performance 
measurement  project  (called  "Project  A")  seeks  to  provide  more  definite  linkages 
between  various  recruit  characteristics  and  certain  measures  of  hands-on  performance  on 
specific  critical  job  tasks.8  However,  to  date  no  one  has  attempted  to  obtain  a  broad 
measure  of  actual  performance  in  realistic  situations  directly  linked  to  wartime 
conditions  and  outcomes,  while  relating  that  performance  to  the  educational  background, 
aptitude,  and  job  experience/training  history  of  personnel. 

This  study  aims  to  develop  improved  databases  and  analyses  for  such  broad-based 
performance  assessment.  The  objective  is  to  produce  empirically  based,  quantitative 
estimates  of  the  linkages  among  training  resources,  soldier  characteristics,  and  the  job 
performance  of  crews  and  small  units.  This  Note  describes  such  a  study  in  the  area  of 
Army  air  defense,  one  of  several  special  research  efforts  in  different  functional  areas. 
Other  areas  covered  by  the  series  of  studies  include  communications,  electronic  system 
maintenance,  armored  vehicle  operation,  and  artillery  operations.9 


BASIC  APPROACH 

The  primary  work  of  MOS  24T  is  to  operate  and  provide  unit  maintenance  for  the 
PATRIOT  Air  Defense  Artillery  system.10  This  research  tests  the  principal  operation 
functions: 


7Recent  school  submissions  to  the  TRADOC  Distribution  of  Quality  program,  which 
requires  schools  to  justify  their  quality  needs,  indicate  that  this  type  of  analysis  continues 
to  predominate. 

8Eaton,  Hanser,  and  Shields,  1987. 

9RAND  is  conducting  the  studies  of  communications  and  electronic  system 
maintenance.  Studies  in  the  other  areas  arc  being  carried  out  by  TRADOC,  the  U.S. 
Military  Academy,  and  the  Army  Research  Institute. 

10We  recognize  that  this  test  concerns  only  the  operations  portions  of  the  MOS; 
maintenance  functions,  however,  are  genetically  different  and  are  the  subject  of  a 
separate  RAND  study. 
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•  Tactical  Control  Assistant  (fire  unit  level).  The  ability  to  protect  valuable 
assets,  engage  enemy  aircraft,  and  assist  in  the  protection  of  friendly  aircraft 
during  air  battles. 

•  Tactical  Director  Assistant  (battalion  level).  The  ability  to  direct  fire  units  to 
engage  specific  aircraft  to  protect  assets  and  destroy  enemy  aircraft. 

As  part  of  this  evaluation,  knowledge  of  proper  system  initialization  procedures  for  both 
functions  will  be  assessed. 

The  performance  of  TCA  and  TDA  functions  will  be  evaluated  using  the 
PATRIOT  Conduct  of  Fire  Trainer  (P-COFT),  a  high-fidelity,  computer-driven  training 
device  that  represents  the  principal  functions  of  the  actual  tactical  equipment  The 
P-COFT  contains  display  screens  and  operators’  consoles;  they  show  incoming  aircraft 
(scripted  according  to  a  preestablished  scenario),  safe  passage  corridors,  defended  assets, 
and  other  airspace  and  ground  features  relevant  to  the  fire  unit’s  mission.  During  a 
simulated  air  battle,  the  operators  track  and  engage  aircraft,  and  the  computer  keeps 
records  of  all  switch  actions.  As  we  will  describe  in  detail,  the  records  maintained  by  the 
simulator  will  provide  a  wide  range  of  effectiveness  measures  that  will  be  used  to 
evaluate  the  performance  of  PATRIOT  operators  in  a  realistic,  wartime-like 
environment. 

OUTLINE 

The  remainder  of  this  Note  presents  the  research  approach,  design,  and  schedule 
for  the  test  of  PATRIOT  operations.  Section  II  describes  the  approach  to  be  taken  in  the 
research,  including  a  description  of  the  high-fidelity  simulator  from  which  the 
performance  measures  are  drawn,  the  scope  of  the  tasks  tested,  and  the  personnel 
examined.  Section  III  presents  the  specific  research  design  and  includes  the  detailed 
procedures  for  testing. 
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II.  RESEARCH  APPROACH 


The  research  will  use  an  existing  high-fidelity  simulation  system,  the  PATRIOT 
Conduct  of  Fine  Trainer,  and  associated  written  tests  of  tactical  operations  and 
initialization  procedures  to  examine  the  proficiency  of  air  defense  soldiers  to  perform  the 
PATRIOT  TCA  and  TDA  mission.  The  examinees  will  be  tested  on  their  ability  to 
protect  valuable  assets,  engage  enemy  aircraft,  and  assist  in  the  protection  of  friendly 
aircraft,  both  at  the  fire  unit  and  the  battalion  level  of  operations. 

FUNCTION  *ND  SPECIALTY  TESTED 

Thi.  Air  Defense  Artillery  protects  U.S.  and  allied  assets  from  enemy  air  attack 
and  has  the  further  mission  of  causing  maximum  attrition  of  enemy  aircraft  The  current 
air  defense  inventory  includes  the  Redeye,  Stinger,  Vulcan,  Chaparral,  Hawk,  and 
PATRIOT  systems.  The  PATRIOT  is  the  most  modem  and  automated  of  these  systems. 
Together  with  the  Hawk,  the  PATRIOT  provides  protection  from  the  high-  to  medium- 
altitude  threat  It  is  capable  of  simultaneously  tracking  large  numbers  of  aircraft  and  of 
engaging  and  destroying  them  at  varying  ranges.  U.S.  PATRIOT  battalions  are  located 
either  at  Fort  Bliss,  Texas,  or  in  the  Federal  Republic  of  Germany. 

A  PATRIOT  fire  unit  (or  firing  battery)  includes  a  fire  control  section  and  eight 
launchers.  Each  launching  station  loads  four  missiles.  The  heart  of  the  battery  is  the  fire 
control  section,  which  includes  an  Engagement  Control  Station  (ECS),  antenna  mast 
group,  radar  set,  and  electric  power  plant  The  ECS,  which  is  a  mobile  shelter  mounted 
on  an  M-814  vehicle,  is  manned  by  three  crew  members:  one  24T  enlisted  member  and 
one  14E  officer,  who  together  control  the  air  Mttle;  and  one  31M  enlisted  communi¬ 
cations  operator  (who  is  not  directly  involved  in  the  air  battle).  The  ECS  is  the  only  fire 
unit  equipment  manned  during  tactical  operations.  It  is  capable  of  operation  in 
"autonomous"  mode  (as  a  stand-alone  facility  using  its  own  radar  and  missiles)  and  in 
"centralized"  mode  (under  the  direction  of  a  PATRIOT  battalion  to  which  it  is  connected 
by  voice  and  digital  data  links). 

The  PATRIOT  Battalion  contains  three  firing  batteries,  under  the  direction  of  the 
Information  and  Coordination  Central  (ICC).  The  ICC  is  similar  to  the  ECS  in  general 
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appearance  and  features,  except  that  it  monitors  a  wider  sector  of  operations  and  directs 
the  activity  of  subordinate  ECSs  when  the  battalion  is  operating  in  "centralized"  mode. 

PATRIOT  enlisted  personnel  fall  into  one  of  three  classifications:  MOS  16T, 
PATRIOT  Missile  Crew  Member,  MOS  24T,  PATRIOT  Operator  and  System  Mechanic; 
and  MOS  24T  with  Additional  Skill  Identifier  T5,  Intermediate  Maintenance  Specialist. 
MOS  16T  personnel  are  responsible  for  emplacement  duties  for  the  PATRIOT 
equipment.  MOS  24T  personnel  are  responsible  for  operation  of  the  ECS  and  the  ICC, 
and  for  unit  level  maintenance. 

This  study  concentrates  on  the  operator  duties  of  24T  personnel.  The  key 
functions  are: 

•  Tactical  Control  Assistant  (TCA).  Operating  at  fire  unit  level,  the  TCA 
engages  aircraft  as  they  approach  defended  assets  and  launches  missiles 
against  other  aircraft  when  fulfilling  an  "attrition"  mission,  in  accordance 
with  tactical  standing  operating  procedures  and  instructions  from  higher 
echelons.  The  TCA  also  assists  the  Tactical  Control  Officer  in  identifying 
aircraft,  carries  out  changes  in  airspace  control  methods  when  necessary, 
initializes  the  equipment  before  a  battle,  and  carries  out  emergency 
maintenance  duties. 

•  Tactical  Director  Assistant  ( TDA ).  Operating  at  battalion  level,  the  TDA 
keeps  track  of  aircraft  across  areas  defended  by  the  battalion’s  three  fire  units 
and  directs  fire  units  to  engage  aircraft  in  accordance  with  tactical 
procedures.  He  assists  the  Tactical  Director  in  management  of  the  battalion’s 
assets  during  the  battle  and  performs  other  functions  similar  to  those  carried 
out  by  the  TCA. 

MOS  24T  is  awarded  after  successful  completion  of  37  weeks  of  Advanced 
Individual  Training  (AIT)  at  the  U.S.  Army  Air  Defense  Artillery  School,  Fort  Bliss, 
Texas.  During  this  period,  both  ECS  and  ICC  operation  training  are  provided.1  The 
training  concentrates  on  system  initialization  procedures  and  the  asset  defense  mission. 
After  graduation,  MOS  24T  personnel  receive  unit  training  in  these  areas.  In  addition, 
they  receive  training  in  the  aircraft  attrition  mission  and  in  air  battle  tactics.  Tactics 

'Extensive  unit  maintenance  training  is  also  provided. 
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include  such  topics  as  airspace  control  methods  and  rules  of  engagement.2  These  topics 
are  covered  in  collective  training  at  Fort  Bliss  before  deployment  of  the  unit  to  U.S. 
Army  Europe,  or,  for  soldiers  shipping  directly  to  Germany,  upon  arrival  at  the  32nd 
Army  Air  Defense  Command  (AADCOM).  In  Germany,  24T  personnel  receive 
additional  training  covering  local  procedures  and  priorities. 

TESTING  ENVIRONMENT 

This  study  uses  the  P-COFT,  which  replicates  operation  of  the  Engagement 
Control  Station  and  Information  and  Coordination  Central.  P-COFTs  are  located  at  the 
Air  Defense  Artillery  School  at  Fort  Bliss  and  at  the  32nd  AADCOM  headquarters  at 
Darmstadt,  Germany. 

Description  of  the  P-COFT 

The  P-COFT  is  a  computer-controlled,  high-fidelity  simulation  facility  consisting 
of  eight  operator  consoles.  The  consoles  can  be  operated  independently  to  simulate 
autonomous  ECS  (fire  unit  or  battery)  operations  or  in  netted  configuration  to  simulate 
fully  interactive  battalion  operations.  The  P-COFT  duplicates  the  TCA  and  TDA 
consoles  in  the  tactical  equipment,  consisting  of  a  display  screen  showing  aircraft, 
airspace,  and  ground  features,  along  with  various  panels  of  switches.  The  features 
displayed  on  the  screen  are  affected  by  operator  actions  in  the  same  way  they  would  be 
in  a  real  ECS  or  ICC.  The  P-COFT  is  driven  by  a  master  computer,  which,  with  a  real¬ 
time  interface,  tabulates  the  individual  actions  taken  by  the  operators  and  the  air  battle 
outcomes  resulting  from  these  actions.  These  data  are  recorded  on  a  disk,  which  may  be 
read  later  for  analysis.  On  behalf  of  RAND,  the  P-COFT  software  contractor,  Sanders 
Associates,  Inc.,  has  developed  additional,  special  software  for  this  test  to  tabulate  new 
measures  of  individual  performance  and  air  battle  results.  The  P-COFT  contains  a 
central  system  manager  console  from  which  the  air  battle  scenarios  are  run,  examinees’ 
actions  can  be  monitored,  and  the  system  is  controlled. 

2Airspace  control  methods  apply  to  the  nonelectronic  (passive)  means  of  helping  to 
ensure  that  hostile  aircraft  are  engaged  and  that  friendly  aircraft  are  not.  They  include 
factors  such  as  safe  passage  corridors  (volumes  of  air  space  in  which  it  is  safe  to  fly); 
weapon  control  volumes  (volumes  of  airspace  within  which  certain  engagement  rules 
apply);  and  speed  or  altitude  restrictions.  The  rules  of  engagement  specify  the  tactical 
operating  procedures,  i.e.,  what  is  to  be  engaged  and  what  is  not,  based  on  the  airspace 
control  criteria,  electronic  identification  results,  and  other  factors. 
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RAND  and  the  Air  Defense  Artillery  School  have  designed  a  test  of  TCA  and 
TDA  functions  for  the  ECS  and  the  ICC,  respectively.  It  allows  us  to  examine  how 
differences  in  personnel  quality,  job  experience,  and  training  background  affect  the 
execution  of  these  functions  and  the  outcomes  of  air  battles. 

Rationale  for  Using  the  P-COFT 

We  selected  the  P-COFT  as  the  testing  medium  first  because  it  provides  realistic 
simulation  of  TCA  and  TDA  duties,  for  which  the  actions  and  outcomes  can  be  measured 
objectively  and  precisely  by  computer.  Second,  the  results  of  a  given  simulation  can  be 
translated  directly  into  air  battle  outcomes.  Third,  the  computer  records  can  provide 
many  details  on  the  process  of  the  battle,  including  the  specific  procedural  steps  that  may 
contribute  to  desirable  or  undesirable  results.  Fourth,  unit  and  battalion  level  operations 
can  be  measured  without  use  of  tactical  equipment — a  considerable  advantage  given  the 
small  number  of  ECSs  and  ICCs  in  existence  and  the  great  demand  for  them.  Fifth,  the 
P-COFT  permits  testing  in  the  same,  controlled  environment  for  all  soldierr  ’>oth  at  Fort 
Bliss  and  in  the  32nd  AADCOM. 
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III.  STUDY  DESIGN 


TESTING  COMPONENTS 

We  have  designed  several  procedures  to  assess  capability  to  perform  critical 
duties  that  must  be  accomplished  in  wartime.  The  entire  testing  procedure  will  be 
administered  and  supervised  by  experienced  air  defense  and  RAND  personnel. 

Simulated  Air  Battles 

Four  20-minute  air  battles  will  be  simulated.  The  scenarios  have  been  developed 
especially  for  this  test  by  subject  matter  experts  at  Fort  Bliss  and  the  32nd  AADCOM 
acting  in  consultation  with  RAND.  The  scenarios  are  designed  to  be  realistic  with 
respect  to  required  wartime  operations  and  events.  They  have  also  been  designed  to 
provide  a  meaningful  test  of  the  performance  of  air  defense  missions  by  both  recent  AIT 
graduates  and  experienced  unit  members.  The  flight  events  scripted  into  the  scenarios 
represent  the  challenges  that  can  be  expected  in  wartime.  They  include  features  such  as 
masking;  jamming;  varying  aircraft  headings,  formations,  speeds,  and  altitudes;  air  to 
surface  missiles;  and  aircraft  with  varying  degrees  of  identification  equipment  problems, 
among  other  features.1 

Area  Defense  Scenario.  The  area  defense  scenario  typifies  operations  required 
during  wartime  in  forward  areas.  The  air  defense  mission  is  to  destroy  the  maximum 
number  of  enemy  aircraft  (consistent  with  prescribed  tactical  procedures  and  rules  of 
engagement,  including  requirements  for  self-defense).  The  scenario  begins  in 
centralized  mode.  During  this  period,  the  TCA  receives  from  the  ICC  specific 
engagement  and  tactical  instructions  that  must  be  properly  executed.  Subsequently,  he  is 
instructed  to  change  to  autonomous  operation  of  the  ECS.  During  the  autonomous 
period,  he  is  solely  responsible  for  his  actions.  As  is  true  for  all  the  scenarios,  the 

'For  example,  aircraft  may  enter  a  PATRIOT  unit’s  area  on  various  headings  (some 
threatening  defended  assets,  some  not),  they  may  turn  or  change  altitude  unexpectedly, 
they  may  fly  within  or  outside  various  corridors  and  designated  airspace  volumes,  and 
they  may  be  “masked"  by  flying  behind  terrain  features  such  as  mountains.  The  RAND 
scenarios  have  been  specifically  designed  to  represent  these  and  other  complex  aircraft 
behavior  patterns  that  are  realistically  expected  in  wartime  and  that  pose  challenges  to 
missile  system  operators  who  must  keep  track  of  numerous  events  occurring 
simultaneously. 
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examinee  receives  information  concerning  his  mission  and  the  tactics  to  be  followed  at 
the  beginning  of  the  scenario.  To  provide  a  standardized  (and  realistic)  protocol,  the 
examinee  is  instructed  to  use  the  automatic  aircraft  identification  mode  and  to  assume 
that  the  identifications  are  valid.  (“Automatic"  in  this  context  means  that  the  computer 
identifies  an  aircraft  as  friendly,  hostile,  or  unknown  based  on  calculations  from  various 
input  evaluation  criteria.) 

Point  Defense  Scenario.  The  point  defense  scenario  typifies  operations 
required  during  wartime  in  rear  areas.  The  air  defense  mission  is  to  defend  the  assets 
assigned  to  the  examinee’s  fire  unit,  such  as  air  bases,  built-up  areas,  storage  facilities, 
and  other  high-priority  locations.  The  scenario  is  run  in  autonomous  operations. 

Halfway  through  the  scenario,  the  TCA  is  informed  that  his  TCO — the  officer  with 
whom  he  is  fighting  the  air  battle — has  died  and  that  he  must  now  also  assume  the 
officer’s  function  of  protecting  friendly  aircraft  (for  which  he  has  been  trained).  During 
this  period,  he  receives  two  instructions  to  carry  out  changes  in  the  procedures  for 
identifying  hostile  and  friendly  aircraft.  The  first  tells  him  to  change  the  current  IFF 
codes  to  an  alternative  set.  The  second  tells  him  that  certain  features  of  the  airspace 
control  methods  have  been  compromised;  he  must  now  manually  identify  aircraft  to 
ensure  their  proper  identification  as  hostile  or  friendly.2 

Mixed  Scenario.  This  scenario  simulates  a  wartime  situation  in  which  the 
primary  mission  is  to  destroy  enemy  aircraft  and  the  secondary  mission  is  to  defend  an 
air  base  assigned  to  the  fire  unit.  Again,  the  missions  must  be  carried  out  in  a  manner 
consistent  with  the  tactical  instructions  provided  to  the  examinee  just  before 
commencement  of  the  air  battle.  The  scenario  is  run  in  autonomous  operations. 

Battalion  Scenario.  The  battalion  scenario  typifies  ICC  operations  required 
during  wartime.  The  air  defense  mission  is  to  assign  individual  enemy  aircraft  to  the  fire 
units  under  the  control  of  the  ICC  and  order  their  engagement.  The  order  includes  the 
method  of  fire — the  number  of  missiles  to  be  fired  and  the  timing  of  the  launches.3  The 

2In  this  situation,  the  operator  must  disregard  the  identifications  generated  by  the 
computer  for  aircraft  within  the  compromised  area;  instead,  he  must  exercise  his 
knowledge  of  airspace  control  methods  to  make  identifications  himself  according  to 
doctrine  and  prescribed  tactical  procedures. 

3The  appropriate  number  of  missiles  depends  on  the  number  of  aircraft  in  a  confined 
airspace  and  the  degree  of  threat  posed  by  the  aircraft,  among  other  things.  The  more 
missiles  fired,  the  greater  the  probability  of  destroying  the  targets;  but  it  is  important  to 
conserve  the  supply  of  missiles,  especially  for  self-defense. 
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orders  must  be  issued  in  accordance  with  the  battalion’s  mission  and  the  specified  tactical 
instructions.  In  this  scenario,  the  mission  is  asset  defense. 

Written  Tests 

In  addition  to  being  tested  on  their  performance  in  fighting  the  simulated  air 
battles,  the  examinees  will  be  required  to  complete  written  tests.  Unit  members  will 
complete  one  test  measuring  their  knowledge  of  proper  tactical  operating  procedures  and 
a  second  test  measuring  their  knowledge  of  system  initialization  procedures  at  the  unit 
and  battalion  levels.4  Soldiers  just  concluding  AIT  will  complete  only  the  initialization 
test,  since  tactics  are  essentially  taught  in  unit  training  procedures.  Each  test  will  consist 
of  25  multiple  choice  questions.  The  tactical  test  questions  are  drawn  from  official  sets 
of  questions  used  to  test  unit  members’  tactical  qualification  for  TCA  duties.  The 
initialization  test  is  drawn  from  official  items  used  to  assess  comprehension  of 
initialization  procedures  during  AIT. 

The  examinees  will  also  complete  a  background  questionnaire  that  collects 
information  on  their  education,  training  history,  unit  assignments,  history  of  job 
responsibilities,  and  air  defense  experience.  This  infoimation  will  be  used  to  supplement 
official  records  concerning  their  performance  on  the  Armed  Services  Vocational 
Aptitude  Battery. 

EXAMINEES  AND  SCHEDULE 

All  MOS  24T  soldiers  at  skill  levels  1  and  2  will  be  tested,  including  personnel 
concluding  AIT  and  the  members  of  all  PATRIOT  units  stationed  both  in  the  continental 
United  States  and  Europe.5  The  test  excludes  skill  levels  3  and  4  (senior  personnel) 
because  our  primary  concern  is  with  the  required  quality  of  recruits.  The  current 
distribution  of  MOS  24T  personnel  worldwide  is  provided  in  Table  1.  Table  2  shows  the 
AFQT  Category  distribution  for  MOS  24T.  Finally,  Table  3  shows  scheduled  graduation 
dates  for  the  AIT  classes  that  conclude  during  the  testing  period  and,  thus,  will  be 

initialization  consists  of  entering  into  the  computer  parameters  that  govern  air  battle 
operations,  such  as  terrain  features,  airspace  control  methods,  and  codes  for  electronic 
identification  of  friendly  and  hostile  aircraft. 

5Soldicrs  in  the  junior  grades  (E-l  through  E-4)  are  at  skill  level  1 ;  they  typically  have 
one  to  four  years  of  service.  Soldiers  at  grade  E-5  are  at  skill  level  2  and  normally  in 
their  second  term  of  enlistment. 


-12- 


Table  1 

WORLDWIDE  DISTRIBUTION  OF  MOS  24T  PERSONNEL  BY  GRADE* 


Command 

E1-E3 

E4 

E5 

E6 

E7 

E8-E9 

Total 

USAREUR 

18 

115 

34 

74 

42 

15 

298 

FORSCOM 

58 

35 

29 

35 

23 

4 

184 

TRADOC 

7 

29 

24 

87 

57 

14 

218 

AMC 

0 

15 

6 

9 

7 

3 

40 

Others 

0 

1 

0 

1 

0 

0 

2 

Total 

83 

195 

93 

206 

129 

36 

742 

'Grades  E1-E4  correspond  lo  skill  level  1,  grade  E5  corresponds  to  skill  level  2, 
grade  E6  to  skill  level  3,  grade  E7  to  skill  level  4,  and  grades  E8-E9  to  skill  level  5. 


Table  2 

AFQT  CATEGORY  DISTRIBUTION  OF  MOS  24T 


AFQT 

Group 

AFQT  Range 
(percentiles) 

Percent  of  Total 

Skill  Levels 
1-2 

Skill  Levels 
3-5 

All 

Skill  Levels 

I 

93-99 

6.2 

9.8 

7.6 

II 

65-92 

47.0 

37.9 

43.5 

IIIA 

50-64 

24.7 

20.6 

23.1 

IIIB 

31-49 

19.3 

23.7 

21.0 

IV 

10-30 

2.8 

8.0 

4.8 

included  in  the  test.  AIT  students  will  be  tested  after  they  have  received  all  applicable 
training  instruction,  near  the  end  of  their  AIT  course.  Unit  members  will  be  scheduled  to 
accommodate  unit  training  activities  and  exercises. 

Final  phases  of  software  testing  and  procedural  pretesting  were  completed  during 
the  months  of  September  to  November  1988.  Testing  begins  in  December  at  Fort  Bliss 
and  in  January  in  Germany.  At  Fort  Bliss,  we  expect  to  test  all  graduating  AIT  students 
in  MOS  24T  and  all  MOS  24T  personnel,  skill  level  1  and  2,  in  school  support  and 
Forces  Command  units  during  the  period  from  December  1988  through  June  1989.  In 
Germany,  unit  members  will  be  rotated  into  the  P-COFT  facility  at  Darmstadt  in  groups 
of  approximately  four  persons  per  day,  making  it  possible  to  test  one  battalion's  24T 
personnel,  skill  level  1  and  2,  in  two  weeks.  If  this  schedule  can  be  maintained,  the 
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Table  3 

AIT  GRADUATION  DATES  FOR  MOS  24T 


Class* 

Number  of 
Students 

Graduation 

Date 

502-88 

16 

7  Dec  88 

503-88 

13 

7  Dec  88 

4-88 

15 

24  Jan  89 

504-88 

16 

22  Feb  89 

505-88 

16 

22  Feb  89 

506-88 

16 

16  Mar  89 

507-88 

14 

21  Apr  89 

509-88 

16 

23  Apr  89 

1-89 

12 

20  Jun  89 

2-89 

13 

20  Jun  89 

8-88 

11 

06  Jul  89 

10-88 

15 

06  Jul  89 

*500  series  classes  were  planned  after  the  initial 
budget  cycle.  In  other  respects,  they  are  the  same 
as  the  lower  number  classes.  Classes  1-89  and 
2-89  have  a  shorter  program  of  instruction. 


testing  in  Germany  could  be  completed  by  April  1989.  The  result  will  be  a  sample  size 
of  up  to  300  unit  personnel  and  150  AIT  students.6 


SPECIFIC  PROCEDURES 

Two  people  will  be  tested  simultaneously.  Each  pair  of  examinees  initially  will  be 
given  a  short  briefing  describing  the  purpose  and  procedures  of  the  test  and  assuring 
i/iem  of  the  confidentiality  of  the  results.  They  will  then  be  asked  to  complete  the  short 
background  questionnaire. 

To  facilitate  the  execution  of  the  testing  procedures  and  to  prevent  possible 
cooperation  between  the  pair  of  examinees,  the  timing  of  the  individual  components  of 
the  test  will  be  staggered.  At  no  point  will  the  two  examinees  complete  the  same  written 

6For  AIT  personnel,  we  are  not  certain  of  the  final  number  of  students  who  will 
actually  graduate  or  the  number  who  will  be  tested  in  the  20  June  cohorts  (since  they 
were  scheduled  for  a  shorter  program  of  instruction),  so  the  total  number  of  examinees 
should  be  about  150.  For  unit  personnel,  there  is  uncertainly  depending  on  the  changing 
number  of  people  who  are  assigned  to  actual  unit  duties  as  24Ts  and  the  fraction  of 
personnel  who  are  actually  available  for  ;ting.  Some  personnel  will  be  acting  in  jobs 
that  may  make  testing  inappropriate  (e.g.,  as  instructors,  aides,  or  workers  in  combat 
development  or  acquisition  jobs)  and  some  will  be  in  transit  or  otherwise  unavailable. 


-14- 


test  or  fight  the  same  air  battle  simultaneously.  The  complete  testing  procedure  will 
require  three  hours  per  pair  of  examinees. 

One  of  the  examinees  will  begin  by  taking  the  simulated  air  battle  portion  of  the 
test.  After  concluding  all  of  the  simulated  air  battles,  he  will  be  asked  to  complete  the 
written  initialization  test  and,  if  a  unit  member,  the  tactical  operations  test  The  second 
examinee  will  complete  the  written  test(s)  before  beginning  the  first  simulated  air  battle. 
This  procedure  ensures  that  no  component  of  the  test  is  completed  simultaneously  by  the 
two  participants.  The  order  of  the  air  battles  will  be  the  same  for  all  examinees:  area 
defense  scenario,  mixed  mission  scenario,  point  defense  scenario,  and  battalion  (ICC) 
scenario.  The  order  is  based  on  providing  continuity  and  an  increasing  level  of 
difficulty.7  Just  before  the  beginning  of  each  scenario,  the  examinee  will  be  provided 
with  written  instructions  concerning  his  mission  and  the  applicable  air  defense  conditions 
and  tactics.  The  scenarios  have  been  written  to  allow  the  participants  one  minute  to  set 
up  their  consoles  before  the  first  engagcable  aircraft  appear. 

PERFORMANCE  MEASURES  AND  ANALYSIS 

The  P-COFT  facility  computer  will  tabulate  all  air  battle  switch  action  and 
outcome  records  automatically.  RAND  will  develop  additional  software  to  analyze  these 
records. 

Table  4  shows  the  measures  of  effectiveness  that  will  be  used  to  evaluate  the 
performance  of  the  examinees.  The  measures  are  divided  into  primary  and  supporting 
measures.  The  primary  measures  reflect  the  direct  outcomes  of  the  air  battles,  such  as 
asset  damage  and  specific  aircraft  engagements.  The  supporting  measures  reflea  the 
actions  and  knowledge  that  underlie  success  on  the  primary  measures.  Table  5  lists  the 
specific  test  faaors  that  will  be  evaluated  and  relates  them  to  the  applicable  measures  of 
effectiveness.  Each  factor  will  be  evaluated  separately  for  each  scenario  and  examinee. 

We  intend  to  model  the  effects  of  various  soldier  charaaeristics  on  these  multiple 
performance  measures  using  multivariate  methods  such  as  regression  analysis  and 
analysis  of  covariance.  Although  the  exact  specification  of  a  model  and  the  functional 

7The  increasing  level  of  difficulty  and  order  of  the  scenarios  may  facilitate 
performance  in  the  later  air  battles  to  a  limited  extent  for  persons  who  have  not 
performed  as  TCA/TDAs  recently  because  of  practice  effects.  In  our  opinion,  and  by 
design,  this  is  realistic  because  such  persons  would  be  given  some  opportunity  to  practice 
before  participating  in  a  real  battle  if  at  all  possible. 
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Table  4 

MEASURES  OF  EFFECTIVENESS  FOR  PATRIOT 
OPERATOR  PERFORMANCE 


Primary  Measures 

A.  Success  in  TCA  point  defense  mission 

B.  Success  in  TCA  area  defense  mission 

C.  Success  in  TDA  mission 

D.  Identification  of  friends  and  foes 

Supporting  Measures 

E.  Compliance  with  tactics/doctrine 

F.  Compliance  with  communications  received 

G.  Knowledge  of  initialization  procedures 


form  cannot  be  determined  in  advance  of  examining  the  data,  we  expect  to  include  as 
predictor  variables  an  array  of  test  scores  such  as  AFQT  and  Army  aptitude  indices; 
educational  level  and  years  of  education;  training  background;  experience  factors  such  as 
time  since  AIT,  time  in  MOS,  location  of  service,  and  type  of  unit  (e.g.,  PATRIOT  and 
other  air  defense  units);  and  specific  job  history  variables  such  as  extent  and  recentness 
of  previous  service  as  a  TCA,  TDA,  and  maintainer.  These  data  will  be  obtained  where 
possible  from  official  records,  such  as  accession  testing  files  of  the  Defense  Manpower 
Data  Center  and  Army  records  from  the  Enlisted  Master  File,  supplemented  by  self- 
reported  questionnaire  data  obtained  directly  from  the  examinees.  With  this  broad  array 
of  factors  in  the  analysis,  we  expect  that  extraneous  effects  will  be  well  controlled  and 
that  the  analysis  will  permit  fairly  precise  estimation  of  the  effects  for  personnel  quality. 
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Tablc  5 


SPECIFIC  TEST  FACTORS  RELATED  TO  MEASURES  OF  EFFECTIVENESS  (MOEs) 


MOEs 

Primary 

Supporting 

Factors 

A 

B 

c 

D 

E 

F  G 

Scenario  Tests  (Forward,  Mixed,  Rear,  Battalion) 

Number  of  hits  to  each  asset  by  priority  of  asset 

X 

X 

Number  of  hits  to  ECS 

X 

X 

X 

Number  of  hostile  aircraft  engaged  (I  AW  tactics 

X 

X 

X 

X 

and  not  IAW  tactics)* 

Number  of  friendlies  engaged 

Total  missiles  used  and  missile  conservation 

X 

X 

X 

X 

X 

Fire  patterns  IAW  tactics  and  not  IAW  tactics 

X 

(number  of  shoot-look-shoot,  ripple,  salvo  by  type 
of  target — e.g.,  raid  size,  threat  code  level) 

Tactical  correctness  of  position  of  aircraft 

X 

and  flight  path  at  launch 

Engagement  of  specified  ATC  targets  ar  slow 

X 

speed  targets 

Console  setup 

Execution  of  ceas  fire,  hold  fire,  and  engagement 

X 

X 

commands 

Success  in  manual  aircraft  identification  procedures 

X 

X 

X 

Execution  of  IFF  code  change 

X 

X 

Wrilter  "rv  _> 

.  >re  on  written  initialization  test  adapted  from  the 

X 

an  of '  'traction 

Sc  on  32nd  AADCOM-adaptcd  TCA  qualification 

X 

X  X 

test  (units) 

*IAW  =  "in  accordance  wiih." 
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